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(57) Decoders A and B decode MPEG-2 bitstreams 
A and B. A switch (SI) switches from decoded stream A 
to decoded stream B to achieve a splice. The resulting 
bitstream is re-encoded in an encoder 4. A new GOP is 
defined beginning at the splice. The new GOP is 
defined by picture type decision rules which may have 
the effect of changing the GOP compared to the GOPs 
of streams A and B. The new GOP provides a prediction 
of the position in stream B where the occupancy value 



of stream C should coincide with that of B. A target for 
the new number of bits in the new GOP is calculated 
dependent on the difference between the occupancy 
value of stream C at the splice and a prediction of the : 
occupancy of stream B at the predicted position. The; 
occupancy value of stream C is controlled In accord- 
ance with the target so that it tends towards the occu- 
pancy value of stream B at the predicted position. 
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Description 

BACKGROUND OF THE INVENTION 
Retd of the invention 

//[OOOI] The present invention relates to a signal processor, a method of signal processing, and a computer program 
/product for Implementing the method. The invention concerns joining digital bit streams which are compressed: that is 
referred to herein as splicing. Embodiments of the invention described herein are concerned with splicing digital video 
io bitstreams which are compressed according to the MPEG-2 standard. 

"• 4 .u . ' 

- r * * 

Description of the Prior Art 



[0002] The invention and its background will be discussed by way of example with reference to MPEG-2 video bit- 
streams. However the invention is not limited to MPEG-2. 

'^ [0003] MPEG-2 is well known from see for example ISO/IEC/13818-2, and will not be described in detail herein. 
Sgllcing of video is well known. It is used in editing video. Splicing analogue signals is relatively straightforward and can 
• . be done at the boundary between adjacent frames, because each analogue frame contains the whole of the video infor- 
mation of that frame independently of other frames. Splicing can be done similarly in the digital domain for both com- 
20 * /pressed and uncompressed video data if all frames contain the whole video information of the frame. 

[0004] MPEG-2 compressed video comprises groups of I and P and/or B frames known as GOPs, Groups of Pic- 
. tures. I, P and B frames are well known. An I or Intra-encoded frame contains all the information of the frame independ- 
ently of any other frame. A P frame in a GOP ultimately depends on an I frame and may depend on other P frames. A 
B frame of a GOP ultimately depends on an l-frame and may depend on P frames in the GOP, but not on another B 
25 frame. 

: : [0005] A GOP typically comprises 1 2 or 1 5 frames comprising at least one I frame and several P and B frames. To 
'/ correctly decode a GOP requires all the frames of the GOP, because a large part of the video information required to 
decode a B frame in the GOP is in a preceding and/or succeeding frame of the GOP. Likewise a large part of the video 

V * information required to decode a P frame is in a preceding frame of the GOP. 

30 [0006] Thus if two different bit streams are spliced together in the compressed domain, the information necessary 

r * :to,decode frames each side of the splice point is likely to be lost. 

[0007] Many papers have been written concerning the splicing of compressed bitstreams, which is a well known 
problem in MPEG. A paper “Flexible Switching and editing of MPEG-2 Video Bitstreams“ by P.J. Brightwell, S.J. Dancer 
and M.J. Knee was published in “Atlantic Technical Papers 1996 / 1 997 " the preface to which is dated September 1997 . 
35 [0008] The paper discusses the problems of splicing MPEG-2 Video Bitstreams. Two bitstreams A and B to be 

spliced are decoded in respective decoders. A coder is switched from the decoder of A to the decoder of B at the splic- 
ing point. It discloses that near a splicing point where a bitstream A is replaced by a bitstream B, the following modifi- 
% cations are made. 

'■ [0009] "The picture type may be changed to provide a more suitable refresh strategy around the switch point In the 
example below, the first P-frame in bitstream B after the switch is converted to an l-frame to provide a full refresh early 
in the new scene. Also, bitstream A contains an l-frame just before the switch point - as this is unnecessary, it is recoded 
as a P-frame to save bits. 
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55 [0010] Prediction modes and motion vectors may require modification to take into account any changes in the pic- 

ture type on recoding, or to prevent any predictions being made across the switch on recoding. In the example above, 
macroblocks that originally used forward or bi-directional prediction for the B-frame following the switch point will be 
recoded using intra mode and backward prediction respectively. In addition, vectors are required for the l-frame that is 
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recoded as a P-frame - these can be estimated from the vectors in surrounding frames, or taken from 1-frame conceal- 
ment vectors that many MPEG-2 bitstreams carry. 

[0011] The quantisation parameters will be changed as part of the recoder's rate control strategy. As in a convert^? 
tional coder, this aims to control the buffer trajectory of a downstream decoder to prevent under- or overflow, and to 
5 maintain the picture quality as high as possible. In addition, the rate control algorithm for the ATLANTIC switch uses the ^ 
vbv_delay values in bitstreams A an B (which are carried in the info-bus) to make the buffer trajectory for the switched 
bitstream identical to that for bitstream B (i.e. the one being switched to) at some future time. Depending on the relative.'/ 
vbv_delay values, this may happen soon after the switch, or a recovery period of a few GOPs may be required. When : : 
it has been achieved, the recoder's quantisation parameters are locked to those of bitstream B, and the switch becomes 
io transparent. 

[0012] The quantisation parameters may also be changed to take advantage of an effect known as temporal mask- 
ing. This refers to the eye’s inability to see moderate or even large amounts of noise around a scene change - typically 
5 dB of degradation in the frame after the switch cannot be seen - and allows the number of bits used for the frames 
* very dose to the switch point to be reduced, allowing a shorter recovery period." , 

is [0013] "Vbv-delay values* are measures of the number of bits in the buffer of the down stream decoder. The man- ^ ' 
ner in which the "buffer trajectory* for the switch bitstream is made identical to that for bitstream B is not disclosed In the 
paper. Also the time scale over which that happens may be "soon after the switch* or after a few GOPs. 

SUMMARY OF THE INVENTION 

20 ’ 

[0014] It is desirable to make the buffer trajectory for the switched bit stream identical to that for the bitstream B in 
•f a more predictable time-scale, preferably one GOP. 

[0015] According to one aspect of the present invention, there is provided a signal processor for splicing a com- 
pressed bitstream B to a compressed bitstream A, comprising 

25 

means for decoding the bitstreams A and B, means for switching from decoded bitstream A to decoded bitstream ' 

B to splice bitstream B to bitstream A to produce a spliced bitstream C, encoding means for re-encoding the spliced , 
bitstream C for supply to a downstream decoder having a downstream buffer, wherein the encoding is controlled by 
control means to produce a prediction of the position on bitstream B at which occupancy of the downstream buffer Yzi 
30 by spliced bitstream C equals what would have been the occupancy by bitstream B alone, and - 

the value of occupancy for bitstream C is controlled between the splice point and the said predicted position Iri 
accordance with the difference between the occupancy value of stream A immediately before the splice and a pfd^ 
diction of the occupancy value of bitstream C at the said predicted position. ~ ' 



35 [0016] Thus by providing predictions of both the position at which occupancy of stream C equals that of B, and of 

the occupancy value, and controlling the occupancy value in accordance with the predictions, lock tends to be achieved 
more reliably in a more predictable time scale. 

[0017] Another aspect of the present invention provides a signal processing method for splicing a compressed bit-! ; . 
stream B to a compressed bitstream A, comprising ' 

40 . .. -• 

decoding the bitstreams A and B, switching from decoded bitstream A to decoded bitstream B to splice bitstream ^ 
B to bitstream A to produce a spliced bitstream C, re-encoding the spliced bitstream G for supply to a downstream 
decoder having a downstream buffer, wherein the encoding is controlled to produce a prediction of the position on 
bitstream B at which occupancy of the downstream buffer by spliced bitstream C equals what would have been the 
45 occupancy by bitstream B alone, and 

the value of occupancy for bitstream C is controlled between the splice point and the said predicted position In 
accordance with the difference between the occupancy value of stream A immediately before the splice and a pre- 
diction of the occupancy value of bitstream C at the said predicted position. 



so [001 8] Yet another aspect of the present invention provides a computer program product arranged to implement the! 

method of said another aspect when run on a programmable signal processor ' 

BRIEF DESCRIPTION OF THE DRAWINGS ‘ ’ 



5s [0019] For a better understanding of the present invention, reference will now be made by way of example to the. 
accompanying drawings, in which: 



Figure 1 is a time chart illustrating the splicing of a bitstream B 0 to a bitstream Aq in accordance with an example 
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v * of the present invention; 

Figure 2 illustrates a portion R of the chart of Figure 1 in more detail; 

‘ Figure 3 is a schematic block diagram of an illustrative signal processor according to the present invention; 

-Sir*' FlQure 4 shows illustrative GOPs in display order and the application of illustrative picture type decisions; 

5 Figure 5 shows the GOPs of Figure 4 in processing order; 

J;; , Figures 6, 6A to 6D, show VBV occupancy for the GOPs of Figure 4 for one embodiment of the invention; 

*V- v ^ Rgures 7A and 7B show the VBV occupancy graphs of Figures 6A to D together with Remain_bit_GOP; and 

Figures 8A and B shows illustrative values of VBV occupancy and Remain J)it_GOP for the GOPs of Figure 4 for 
another embodiment of the invention. 



io . > 




is : [Q020] I frames have the following transcoding parameters which are well known in MPEG:- 

: ': v , DCTJType, Q and Q_Matrix which are reused in embodiments of the present invention described hereinbelow. 

. : These parameters are reused in the recoding of I frames with reuse of parameters. These parameters are recalcu- 
lated when fully recoding I frames. 

20 P and B frames have the parameters DCTJType, Q, Q_Matrix, Pred_Type, MB_Mode and Motion Vectors. These 

parameters are reused in the recoding of P and B frames with reuse of parameters. These parameters are recal- 
culated when fully recoding all frames. 



Overview 

2S : . 

[0021] Referring to Rgures 1, 3 and 4, two bitstreams Aq and B 0 are MPEG-2 encoded with GOPs comprising 12 
frames. In this example the bitstreams A 0 and B 0 have the same GOP structure: 

IBBPBBPBBPBB 

30^:S : 

as shown in Figure 4. However the bitstreams may have any other GOP structure allowed by MPEG-2. The two 
bitstreams Aq and B 0 may have different GOP structures. For ease of explanation it is assumed the bitstreams Aq and 
B 0 have the same GOP structure as shown in Figure 4. 

[0022] It is desired to replace bitstream Aq by bitstream B 0 . As shown in Figure 1 , initially Aq is provided to the proc- 
35 essor. It is routed in the processor P of Figure 3 from input A 0 to contact Aq of switch S} where it is fed, unchanged to, 

: for example, a downstream decoder 2. In decoder 2 it is decoded for display. Downstream decoder 2 may be in, for 

example, a domestic television receiver Processor P may be in a studio. 

■* [0023] When an operator decides to splice bitstreams B 0 and Aq, the operator operates the switch SI and a switch 
0 S2 so that Aq is routed through a decoder A, an encoder 4 and via contact C of switch SI to the downstream decoder 2. 
40 [0024] The bitstream Aq is decoded in decoder A and re-encoded in encoder 4. The MPEG-2 parameters are 

. ; derived from the decoder A by a control processor 6 and re-used in the encoder 4, so that the decoding and re-encoding 
.. is as loss-less as possible. {There may be some loss because the DCT rounding process can cause the DCT process 
to be not transparent.) 

[0025] Bitstream B 0 input at input B 0 is also decoded in a decoder B. The decoded bitstream B 0 is supplied to con- 
45 tact B of switch S2. 

[0026] The processor P has sufficient storage (not shown) associated with decoders A and B to store for example 
30 compressed frames. 

[0027] A splice point SPLICE is chosen. At the splice point switch S2 selects the decoder B and decoded bitstream 
B is fed via the encoder 4 to the contact C of switch SI . In the example of Figures 1 and 4, full recoding, that \s without 
so re-use of the MPEG parameters, begins on the bitstream Aq 5 frames before the splice point SPLICE. The reason for 
this will be explained below. After the splice, bitstream Aq is irrelevant except that some frames of Aq after the splice 
may be needed to decode frames of Aq occurring before the splice. 

[0028] After the splice point SPLICE, the bitstream B 0 is fully recoded for a transition period during which VBVJock 
is achieved as will be explained below. Once VBVJock is achieved, recoding of bitstream B 0 continues but with re-use 
55 of the MPEG parameters derived from the original bitstream B 0 . 

[0029] After a short interval of recoding of B 0 with re-use of the MPEG parameters, switch SI selects contact B 0 
and thus the original bitstream Bq, by-passing the decoder B and encoder C. 

[0030] Rgure 1 shows a period "Reuse “A"" preceding the "full recode" period. Reuse "A" is optional in that period. 
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The original bitstream Aq may be used in that period, unchanged. Also the period of Reuse of "B* after the achievement 
of exact lock as described below is optional. The original bitstream B 0 may be used unchanged. 

[0031] The processor P has delays D A and Dq in the direct signal paths from inputs Aq and B 0 to the contacts- A© 
and B 0 of switch SI for compensating for the signal processing delays in the decoders A and B and the encoder 4 ^ 

5 

Picture Type Decisions ]-i\ 

[0032] The splicing of bitstream B 0 to bitstream Aq disrupt s the GOP structure. Thus the following rules are applied 
in a first method to be described herein below. In a second method they are applied but in modified form as described ’ 
io herein below. 

[0033] The bit stream Aq, in the example of the first method, before the splice is recoded so that: 

(1) the two T or *P' frames before the splice are converted to 'P'; 

is (2) if the last frame before the splice is a 'B‘ frame, it is converted to 'P\ 

The bitstream, in this example Bq, after the splice is recoded so that: 

(3) the first T or ’P* frame after a splice is converted to T; and ■ 

20 (4) if the first GOP af te r the splice and after the application of rule (3) contains less than three 'P* frames, the T / 

frame of the subsequent GOP is converted to 'P 1 , thereby lengthening the GOP. 



25 



[0034] In effect, a new transitional GOP begins with an l-frame at the splice. The new GOP may be different in 
length, e.g. longer or shorter than preceding (and succeeding) GOPs in the bitstream B 0 It may be the same length as 
the original GOPs. The new GOP is in effect a prediction of where VBVJock is to be achieved. The application of these 
rules is shown in Figure 4 at (1), (2), (3) and (4). 

[0035] In Figure 4 ^ 



30 



A is bitstream Aq, 

B is bitstream B 0 , and 
C is the spliced bitstream, 



35 at contact C of switch SI as if the picture type decisions have not been made, and C 1 is the spliced bitstream at 

contact C with the picture type decisions applied to it. 

[0036] By application of rule (2), the 'B* frame of Aq immediately before the splice is convened to P. By application 
of rule (t ) the I frame of A 0 before the splice is also converted to P. 

[0037] By application of rule (3), the first ‘P* frame of stream B 0 after the splice is convened to I in stream C 1 . 

40 [0038] By application of rule (4), the GOP of bitstream Bq after the splice has (after conversion of its first P frame 

to I) less than 3 'P' frames. Therefore the next T frame is convened to 'P'. Application of these rules gives a GOP which ^ 
defines a predicted VBVJock point as will be discussed below. 

Processing Order 

45 

[0039] Figure 4 shows the frames of the bitstreams in the order in which they are displayed or would be displayed. 
The frames are not processed in that order. Figure 5 shows the order in which the frames are processed. For exampld, 
referring to Figure 4 (Display Order) frames 0, 1 and 2 row of bitstream A are shown in that order. Even though the B 
frames 0 and 1 would be displayed before I frame 2, they depend on I frame 2 to be decoded. Thus to decode them I . 
so frame 2 must precede the B frames as shown in Figure 5. Likewise B frames 3 and 4 of Figure 4 depend on P frame 5 • r 
of Figure 4; thus In Figure 5 P frame 5 of Figure 4 becomes P frame 3 preceding the two B frames. 

Constant bit rate 

55 [0040] The example of the processor P of Figure 3, has a constant bit rate. The bitstreams A Q B 0 , have a fixed bit 

rate and the encoder 4 produces at contact C of switch Si a constant bit rate. 
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Downstream Decoder and Buffer 

V : ;[0041] The downstream decoder 2 has a buffer 8. The encoding which takes place in encoder 4 of the processing 
- V js^anranged so that the buffer 8 of the downstream decoder 2 neither underflows nor overflows. Figures 6, 7 and 8 shows 
operation of the downstream buffer 8 of the downstream decoder 2. 

0k [0042] (The encoder 4 has a corresponding buffer and it operates as the inverse of what is shown in Figures 6 # 7 
^ . and 8.) 

.J:-’[0p43] The following are known MPEG rate control parameters. 

■V v 

10 VBV 

- [0044] VBV is video buffer verifier. It is a measure of the number of bits that would be in the downstream buffer 8. 
Remain bitGOP 

y- [0045] Remain_bit_GOP Is a target number for the total number of bits for the remainder of the current GOP. At the 
beginning of a GOP, it is a target for the whole GOP. It reduces as the GOP progresses. 

Complexity X. constants Kp. Kb and N. Np. Nb 

r [0046] 

^ N is the number of pictures in a GOP. 

25 Np is the number of P frames in a GOP. 

:: Nb is the number of B frames in a GOP. 

'* Kp and Kb are 'universal' estimates dependent on quantisation matrices. They (indirectly) define the relative sizes 
36 of 1, P and B frames. Xi, Xp, Xb are "complexity measures" for I, P and B frames. 

[0047] These parameters are used in a known manner to distribute the bits of a GOP amongst I, P and B frames. 
[0048] They are further explained in Test Model 5" published by "International Organisation for Standardisation 
Organisation Internationale De Normalisation, Coded Representation of Picture and Audio Information ISO/I EC 
35 JTC1/SC29/WG1 1/N0400". 

Achieving VBV Lock 

[0049] As discussed above, the downstream buffer 8 must neither underflow nor overflow. In MPEG-2 the buffer is 
40^ normally kept approximately half-full. A discontinuity in the bit stream can make the buffer underflow or overflow. VBV 
v>; is the measure of buffer occupancy. 

[0050] Figures 6, 7 and 8 show, as an extreme case, buffer occupancy VBV for two bitstreams A 0 and Bq- Ao has a 
• typical occupancy and B 0 has unusually high occupancy. 

[0051] In figures 6, 7 and 8, A, B and C indicate buffer occupancies for bitstreams A 0 B 0 and C. R indicates 
45 Remain_bit_GOR 

[0052] In the situation where the bitstream begins with Aq with typical occupancy and B 0 With high occupancy is 
spliced onto Aq at the splice point, it is necessary to: 

a) provide continuity, albeit changing, of VBV occupancy; and 
so 

b) change the VBV occupancy from the value of stream Aq just before the splice to a target value which is the value 
of VBV for stream B 0 . 

[0053] As best shown in Figure 7 or 8, the VBV of bit stream C begins identical to Aq, then changes progressively 
55 ‘ towards the VBV of Bq. 

[0054] The point at which the VBV occupancy of C becomes identical to that of stream B 0 is the VBVJock point. 

[0055] As mentioned above Figures 6, 7 and 8 show occupancy of the downstream buffer 8. To achieve VBVJock 

the encoder 4 is controlled. 
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[0056] Figures 6 and 7 show the second method (referred to herein as “short latency)" of achieving VBV lock. Fig- 
ure 8 shows the first method (referred to herein as "long latency*) which uses the picture type decision rules set out 
above. The first method of Figure 8 will be described followed by the second method. 

s First Method (long latency) 

[ 0057] ^ ’ 

a) The first method uses the picture type decision rules (1) to (4) above unchanged. 
io b) Adjust Remain_bit_GOP. 

A Transitional GOP is defined starting at the splice point SPLICE and ending at a point defined by the picture 
decision rules. 

The length of the transitional GOP (i.e. the number of frames in the GOP) is known from the result of adjusting 
the GOP length using the picture decision rules. [This can be achieved because the processor P stores for example 
is 30 frames as metioned above). V 

The value of Remaln_bit_GOP is reset to zero at the splice point. The value of Remain_bit_GOP Is recalcu- ’ 
iated in normal manner for the transitional GOP following the splice point, and t ; 

a value Extra_bits is added to the result, where Extra-bits is 

20 

[VBVJC_SpIice - VBV_B_Lock] 

where VBV_C_Splice is VBV occupancy of stream C immediately before the splice which is always the end of 
the previous GOP adjusted in accordance with the picture decision rules; and VBV_B_Lock is the VBV occu- 
25 pancy of the stream B 0 at the end of the new transitional GOP. This value is known because the frames of the 

stream B 0 are stored and the length of the new GOP is known. ’ . “ ■ 

Extra-bits can be negative. 

The initial value of Remain_b‘it_GOP calculated in this way is reduced for example by multiplying it by a factor V 
30 a where a is one or less. The factor a is chosen to reduce the value of Remain_bit_GOP by a small amount e.g. . 
5% or less so that spare bits allow additional bits ('stuffing' bits) to be added to achieve exact lock. The need for this 
will be explained below. The factor a is empirically determined. In the following discussion it is assumed to be fixed. 
However it may be varied. 

Remain_bit_GOP is then updated at every frame by the number of bits used to encode that frame up to the 
35 end of the transitional GOP. 
c) Complexity 

Stream Aq is decoded and reencoded before the splice. Thus encoding occurs with complexity values appro- 
priate to stream Ao* However, these values are not appropriate for stream B 0 . Thus before the splice, complexity 
values Xj, Xp, Xq of the I, P and B frames of stream B 0 immediately before the splice are calculated based oh 



X=S.Q 



where 



X = complexity value 



S = number of bits generated by encoding a picture 
Q = average quantisation parameter of all macroblocks in a picture. 

X = S.Q is a standard equation for rate control in MPEG. 

Alternatively, the complexity values may be derived from the first I, P and B frames after the splice or fromTfie 
I, P and B frames nearest the splice. 

At the splice point these complexity values replace the existing values (of stream A 0 ). So after the splice com- 
plexity values appropriate to stream B 0 are used. 

The complexity values control the distribution of bits amongst I, P and B frames. Achieving good subjective 
quality is dependent on the complexity values. 
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;> t 




TO' 



dV Virtual Buffers Modification 

The virtual buffers are used to calculate the reference Q scale for each macroblock. Improvement in quality can 
be gained by setting the virtual buffers to estimated stream B 0 values at the splice point This ensures that the 
resulting Q scales are similar to those used in stream Bq in the previous generation, instead of continuing with the 
Q scales of stream A 0 . 

StreamB virtual buffer values are estimated for the last I, P and B frames before the splice point based on Q. 
Alternatively they are estimated from the first I, P and B frames after the splice or from the I, P and B frames nearest 
the splice. 

For the I frame, the value is calculated as: 

estimated_buf J = (2*Q * bit_rate)/(31 * frame_rate). 



Q is the average quantization parameter. 

This value is then forced at the splice point. 

T5 ; ., _ This is also done for the P and B virtual buffers in same way. 

. * 10058] The control 6 controls the encoding in encoder 4 in accordance with Remain_bit_GOP, between the splice 
point SPLICE and VBVJock so that the occupancy of the downstream buffer 8 follows a continuous but changing tra- 
: jectory from before the splice at bitstream A occupancy to VBVJock at the bitstream B occupancy. The control is also 
20 - used to force the complexity and virtual buffers as described above. 

‘ : [0059] In order to increase occupancy of the downstream buffer as shown in Figure 6, the buffer in the encoder is 
controlled to output pictures with smaller number of bits, so that its occupancy decreases. Pictures with smaller num- 
bers of bits are produced by increased compression/coarser quantisation. 

[0060] If the trajectory is from high occupancy of the downstream buffer to lower occupancy of the downstream 
25 buffer, the encoder is controlled in accordance with the higher Remain-bit-GOP to increase occupancy of its buffer, pro- 
ducing larger pictures by less compression/finer quantisation. 

- [0061] Various modifications may be made to the first method. The VBV values of a bitstream, e.g. stream B tend 

; to maintain roughly constant peak values. Thus instead of finding the peak VBV of stream B at the predicted lock point 
" VBVJock the VBV value of stream B at the I frame closest to the splice point could be used as a prediction of occu- 
30* paincy at VBVJock. 

. [0062] The value of the prediction can be adjusted periodically through the GOP as in the Second Method as fol- 
. lows. 



Second Method (short latency) 

35 

[0063] 

a) The second method uses the picture decision rules (1) to (4) above but with the following modification: 
40 Rule (1) is changed to 

* (1 ’) The last T or *P' frame of the bitstream (A) before the splice is converted to ’P 1 . 



45 The result of the picture decision rules in the example shown in Figures 6A to D is exactly the same as in the 

first method. 

b) Adjust Remain Jrit_GOP. 

In order to increase occupancy of the downstream buffer as shown in Figure 6, the buffer in the encoder is con- 
50 trolled in accordance with Remain_bit_GOP to output pictures with smaller number of bits so that its occupancy 
decreases. Pictures with smaller number of bits are produced by increased compression/coarser quantisation. 

If the trajectory is from high occupancy of the down stream buffer to lower occupancy of the downstream buffer, 
: the encoder is controlled in accordance with the higher Remain J>it_GOP to increase occupancy of its buffer, pro- 

ducing larger pictures by less compression/finer quantisation. 

55 Remain_bit J30P is the target for the number of bits remaining in the transitional GOP as discussed in the first 

method. 

The length of the transitional GOP is known from the result of adjusting the GOP length using the picture deci- 
sion rules modified for the second method. 
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The value of Remain J>'itJ30P is reset to zero at the splice point. The value of Remain J>'rt_GOP is recalcii- 
lated in normal manner for the new GOP following the splice point The value of Remain Jxt_GOP at the splice is 
changed by a value VBV_d'rff where 

5 V B V_diff= VB V_C_sp lice- VB V_B_n ext I or P. ; ^ 

That is, Remain Jjrt_GOP is initially set to the sum of [the normal allocation of bits for the GOP] and [the differ- 15 
ence between (the VBV value of stream C at the splice) and (the VBV value at the first I or P frame following the 
splice of the stream B)]. This is a prediction of the VBV of the bitstream B at the VBVJock point The value of 
io Remain_bit_GOP is then reduced, by a factor a for the transitional GOP as in the first method. 

The factor a reduces Remain J)*rtJ30P for the transitional GOP by an empirically determined amount e.g. 5% 
or less. It is assumed herein that a is fixed. It may be varied. 

Remain JjitJB OP is updated at every frame by the number of bits used to encode that frame. In addition, the 
following update occurs: - 

15 At the splice point VBV_d*iff is based on (VBV_C_splice-VBV_B_next I or P). Subsequently at every succeeds ; 

Ing I and P frame of the stream B, VBV_DIff=VBV_B_currentJ or P — VBV_B_next I or R If VBV_diff Is positive ■ 
the change is divided by the number of l and P frames in the remainder of the transitional GOP and the result added 
to Remain J)it_GOP at each subsequent update. If VBV_diff is negative, Remain J>it_G OP is reduced by VBV_diff. r? r 
In stream C of Figures 7 and 8, Remain J>it_GOP is thus updated every 3 frames. This continues until VBVJock is ■ 

20 achieved. - 

The initial value of Remain-bit GOP is reduced by the factor a. a is chosen to reduce the value of Remain-bit- * 
GOP so that spare bits are available at VBVJock. The spare bits allow additional bits ("stuffing bits") to be added 
to achieve exact lock. The need for this will be explained hereinbelow. 

The differences VBV_diff can be calculated because the frames are stored. 

25 

c) Complexity This is dealt with in the same way as in the first method. 

d) Virtual Buffer Modification This is dealt with in the same way as in the first method. 

30 VBV_Lock_point . 

[0064] This applies to both the first and second methods. {?vr -; 

[0065] The period over which VBVJock is predicted to be achieved is one GOP albeit a GOP the length of which 
may be changed by the picture type decision rules. In this example it is about 23 frames. 

35 [0066] Referring to Figures 2, 4, 5 and 6 assume lock is achieved at l-frame 52 in display order (Figure 2 or 4). In 

fact the VBVJock is achieved in the processing order so that it occurs at reordered I frame 50 (Figure 5). The following 
B frames 51 and 52 in Figure 2 or 5 are fully recoded frames from prior to VBVJock and disturb the lock. Thus the spare 
bits are used at the second B frame 52 to stuff the bitstream to achieve exact lock. 

[0067] If the factor a is one, then no spare bits are available so the system attempts to achieve exact lock at ftfir f ■ : 
40 frame immediately after the end of the transitional GOP. The rate control under or over steers producing usually too 
many bits. Even if exact lock is achieved at the I frame, lock is disturbed at the B frames. Thus, Remain J>’it_GOP is , ; 
reduced by the factor a, and rate control oversteers so that spare bits are available at the end of the GOP. The spare 
bits are used to achieve exact lock at the second B frame. V;* 

[0068] The l-frame 50 is processed by reusing its parameters derived from the original bitstream Bq. After the fully 
45 recoded B frames 51 and 52, re-use of parameters resumes until the original bitstream B 0 is directed directly to the out- 
put of the processor bypassing the decoding and re-coding. 

[0069] This applies to both the first and second methods. 

Modifications 

so - 

. [0070] The discussion hereinabove assumes VBVJock will be achieved at the end of the transitional GOP after the * : 
splice point. VBVJock may not occur in the event of extreme differences between the two bitstreams. 

... [0071] In the second method, the value of Remain JaitJBOP is updated every 3 frames and the process of trying to ^ 
achieve VBVJock progresses until lock is achieved. 

55 [0072] In the first method, within the transitional GOP after the splice point, lock is achieved with the appropriate., 

value of a. a is chosen so that lock is achieved except for the extreme case, a may be variable. 

[0073] The picture type decision rule (4) may be changed to be (4 1 ): 
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(4 1 ) If the first GOP after the splice contains only one 'P' frame, the frame types of the next GOP are altered from; 
\ V 'I to ’P'; and 'P* to T; to give two ’P‘ frames in a GOP. 

*!«• . [0074] This results in two shorter GOPs between the splice point and VBV_lock. 

5 '- - pfl75] In this case Extra bits is divided between the two GOPs in suitable proportion. (Extra-bits could be negative). 
[0076] Some bit streams may comprise only I and B frames or only I and P frames. 

v . Motion Vectors 

w ^6fl77] Motion vectors are regenerated for the frames of the transitional GOP. Alternatively motion vectors could be 
estimated from vectors in neighboring frames. 

[0078] The motion vectors for frames of the transitional GOP may be derived from the corresponding frames of the 
original bitstream A 0 or B 0 . If an I frame is changed to P because of the picture decision rules concealment vectors 
maybe used if they are present: see the Atlantic paper discussed hereinbefore. For other frames, that is frames recoded 
is from P and B frames, vectors are available. However, if a B frame is recoded as P, the vectors may need to be modified 
;; to point to the appropriate frame preceding the frame recoded as P. 

. : / Bit Growth on frames outside transition region 

20 [0079] As shown in Figure 1 after the end of the transitional GOP, the frames of the bitstream B are recoded with 

.i , -.reuse of transcoding parameters until stream B 0 bypasses the decoder B and encoder 4. 

: f0080] The VBV of such recoded frames may not be the same as their original forms. The number of bits may shrink 
or grow because inverse DCT and DCT processes are subject to errors, and because of de-quantization and quantiza- 
tion errors, in decoding and re-encoding. If the shrinkage or growth is excessive the downstream buffer may under-or 
25 over-flow. 

[0081] The frames are monitored. If excessive shrinkage occurs bit-stuffing is used, and if excessive growth occurs 
frames are recoded to maintain occupancy with normal MPEG limits. Preferably encoding reuses the transcoding 
parameters of the I frames to maintain picture quality. 

[&082] The foregoing describes by way of example a signal processor and a method of processing signals. The 
30 . processor, and the method, may be implemented using special purpose hardware which may include a programmable 
V-; ’ computer or be a programmable signal processor. The signal processing method may be embodied in a computer pro- 
gram product, which, when run on the computer or other programmable processor, implements the method. 

- [0083] Although illustrative embodiments of the invention have been described in detail herein with reference to the 
accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that 
35 various changes and modifications can be effected therein by one skilled in the art without departing from the scope 
and spirit of the invention as defined by the appended claims. 

Claims 

40 1. A signal processor for splicing a compressed bitstream B to a compressed bitstream A, comprising 

means for decoding the bitstreams A and B, means for switching from decoded bitstream A to decoded bit- 
stream B to splice bitstream B to bitstream A to produce a spliced bitstream C, encoding means for re-encoding 
the spliced bitstream C for supply to a downstream decoder having a downstream buffer, wherein the encoding 
4S is controlled by control means to produce a prediction of the position on bitstream B at which occupancy of the 

downstream buffer by spliced bitstream C equals what would have been the occupancy by bitstream B alone, 
and 

the value of occupancy for bitstream C is controlled between the splice point and the said predicted position in 
accordance with the difference between the occupancy value of stream A immediately before the splice and a 
so prediction of the occupancy value of bitstream C at the said predicted position. 

2. A processor according to claim 1, wherein said prediction of the occupancy value of stream C is the occupancy 
value of stream B at the predicted position. 

55 3. A processor according to claim 1 , wherein the said prediction of the occupancy value of stream C is the peak occu- 

pancy value of stream B at a point adjacent to the splice point. 

4. A processor according to claim 1 , 2 or 3, for processing bitstreams A and B which are video bitstreams each having 
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sequences of compressed frames each sequence having a predetermined number of frames, and which produces 
a transitional sequence having a number of frames dependent on the position of the spice in the sequence of 
streams A and B following splice point, 

5 the transitional sequence providing the said prediction of the said position. * % >; r 

5. A processor according to claim 4, wherein the transitional sequence is defined by a predetermined set of rules.. 

~ . . V«; 

6. A processor according to claim 5, for processing MPEG bit streams comprising Groups of Pictures (GOPs), and * 

io the transitional sequence is a transitional GOP defined by the said rules. 

7. A processor according to claim 6, wherein the control means calculates for the transitional GOP a target number 
of bits Remain_bit_GOP for the GOP, and controls the value of occupancy for bitstream C within the transitional 
GOP in dependence upon Remain_bit_GOP. 

is 

8. A processor according to claim 7, wherein Remaln_bit_GOP Is reduced by a small amount. 

9. A processor according to claim 8, wherein additional bits are added to the stream C after the position at which the - 
occupancy value for stream C approximately equals that for stream B to achieve equality. 

20 'Y']& 

10. A processor according to any one of claims 6 to 9, wherein the picture type of any frame which is not a B frame : 
immediately before the splice point is changed to P. 

11. A processor according to claim 10, wherein if there are two T or 'P* frames before the splice point they are con- 

25 verted so both are *P' frames. 

1 2. A processor according to claim 1 0 or 1 1 , wherein if the last frame before the splice is a 'B 1 frame, it is changed to a 

'P* frame. • • ; 

30 13. A processor according to any one of claims 6 to 12, wherein the transitional GOP has I and P and/or B frames, and 

wherein the transitional GOP is produced by modifying stream B following the splice point so that the first I or P 
frame after the splice is converted to I. **’ 

14. A processor according to claim 13, wherein if the said transitional GOP has less than three P frames, the I frame 

35 of the subsequent GOP is converted to P. 

15. A processor according to claim 13, wherein if the first GOP after the splice contains only one 'P' frame, the frame 
types of the next GOP are altered from: *1* to ‘P'; and 'P' to T. 

40 16. A processor according to any one of claims 6 to 12, wherein the transitional GOP is produced by modifying thei . ; 

stream B after the splice so that it has “* 

a) an I frame at the beginning of the transitional GOP following the splice point, and, in addition to the said I 

frame, 

45 b) at least three 'P* frames. • 

17. A processor according to any one of claims 6 to 9 and 16, wherein Remain_bit__GOP is calculated as the sum of; 

a) the difference between the values of occupancy of stream C at the splice point and the prediction of the 

so occupancy at the said predicted position; and 

b) the number of bits calculated for the GOP as if the said difference a) was zero. 

18. A processor according to claim 17, wherein the value of Remain_bit _GOP is reduced on every frame by the 
number of bits used to encode that frame. 

55 Xi ‘ 

19. A processor according to any one of claims 6 to 9 and 16, wherein Remain_bit_GOP is initially calculated as the ' 
sum of 
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a) the difference Df between the occupancy value of stream C at the splice and the occupancy value of the 
nearest I or P frame of stream B following the splice; and. 

b) the number of bits calculated for the GOP as if the said difference a) was zero. 

A processor according to claim 1 9, wherein Remain_bit_GOP is subsequently changed at every I and P frame. 

A processor according to claim 20, wherein Remain_bit_GOP is subsequently changed at every I and P frame in 
dependence upon the difference D2 between 

a) the occupancy value of stream B at the current I or P frame; 

b) the occupancy value of stream B at the succeeding I or P frame. 

A processor according to claim 21, wherein, if the said difference D2 is negative, Remain_bit_GOP is reduced by 
the said difference D2. 

A processor according to claim 19, 20, 21 or 22, wherein, if the said difference D1 orD2 is positive, a value E equal 

to the difference Dt or D2 divided by the number of I and P frames remaining in the GOP is calculated and E is 

added to Remain_bit_GOP at each subsequent I and P frame in the GOP. 

20 .-X-. 24. A processor according to any one of claims 19 to 23, wherein the value of Remain_bit _GOP is reduced on every 
/* ; ~ frame by the number of bits used to encode that frame. 




rJ ’***»$7 
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25. A processor according to claim 1 5, wherein a value F is calculated as the sum of; 

25 a) the difference between the values of occupancy of stream C at the splice point and the prediction of the 

r occupancy at the said predicted position; and 

. ; b) the number of bits calculated for the GOP as if the said difference a) was zero, and F is apportioned between 
. the said first and next GOPs. 

* *■ c 

. i :* 

3d 26. A processor according to any one of claims 6 to 25, wherein the complexity values of stream B replace those of 
stream A at the splice point 

27. A processor according to any one of claims 6 to 26, wherein the virtual buffer values of stream B replace those of 
stream A at the splice point. 

35 

28. A signal processing method for splicing a compressed bitstream B to a compressed bitstream A, comprising 



decoding the bitstreams A and B, switching from decoded bitstream A to decoded bitstream B to splice bit- 
stream B to bitstream A to produce a spliced bitstream C, re-encoding the spliced bitstream C for supply to a 
40 downstream decoder having a downstream buffer, wherein the encoding is controlled by to produce a predic- 

.-f; tion of the position on bitstream B at which occupancy of the downstream buffer by spliced bitstream C equals 

what would have been the occupancy by bitstream B alone, and 

the value of occupancy for bitstream C is controlled between the splice point and the said predicted position in 
accordance with the difference between the occupancy value of stream A immediately before the splice and a 
45 prediction of the occupancy value of bitstream C at the said predicted position. 

29. A method according to claim 28, wherein said prediction of the occupancy value of stream C is the occupancy value 
of stream B at the predicted position. 



so 30. A method according to claim 28, wherein the said prediction of the occupancy value of stream C is the peak occu- 
pancy value of stream B at a point adjacent to the splice point. 

31. A method according to claim 28, 29 or 30, for processing bitstreams A and B which are video bitstreams each hav- 
ing sequences of compressed frames each sequence having a predetermined number of frames, and which pro- 
55 duces a transitional sequence having a number of frames dependent on the position of the splice in the sequence 
of streams A and B following splice point, 



the transitional sequence providing the said prediction of the said position. 
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32. A method according to claim 31 , wherein the transitional sequence is defined by a predetermined set of rules. 

33. A method according to claim 32, for processing MPEG bit streams comprising Groups of Pictures (GOPs), and the 
transitional sequence is a transitional GOP defined by the said rules. 

5 * - 

34. A method according to claim 33, wherein a target number of bits Remain_bit_GOP is calculated for the transitiohal ?!' 

GOP and the value of occupancy for bitstream C within the transitional GOP is controlled in dependence upon 
Remain-bit-GOP. • 

, - *.V; 

10 35. A method according to claim 34, wherein Remain-bit-GOP is reduced by a small amount. 

36. A method according to claim 35, wherein additional bits are added to the stream C after the position at which the 
occupancy value for stream C approximately equals that for stream B to achieve equality. 

is 37. A method according to any one of claims 33 to 36, wherein the picture type of any frame which is not a B frame 
immediately before the splice point Is changed to P. 

• ... . - v 

38. A method according to claim 37, wherein the two T or 'P' frames before the splice part are converted so both are :/ 
*P' frames. 

20 )X 

39. A method according to claim 37 or 38, wherein if the last frame before the splice is a 'B' frame, it is changed to a *P ' 
frame. 

40. A method according to any one of claims 33 to 39, wherein the transitional GOP has I and P and/or B frames, and 

25 wherein the transitional GOP is produced by modifying stream B following the splice point so that the first I or P 

frame after the splice is converted to I. 

41. A method according to claim 40, wherein if the said transitional GOP has less than three P frames, the I frame of 
the subsequent GOP is convened to P. 

30 

42. A method according to claim 40, wherein if the first GOP after the splice contains only one 'P 1 frame, the frame 
types of the next GOP are altered from 'I; to 'P* and P* to ‘I*. 

43. A method according to any one of claims 33 to 39, wherein the transitional GOP is produced by modifying the. 

35 stream B after the splice so that it has 

a) an I frame at the beginning of the transitional GOP following the splice point, and, in addition to the said I 
frame, 

b) at least three ’P 1 frames. 

method according to any one of claims 33 to 39 and 43, wherein Remain_bit_GOP is calculated as the sum of;: -: 

a) the difference between the values of occupancy of stream C at the splice point and the prediction of the , .... 
occupancy at the said predicted position; and 

b) the number of bits calculated for the GOP as if the said difference a) was zero. 

method according to claim 44, wherein the value of Remain_bit _GOP is reduced on every frame by the number 
of bits used to encode that frame. 

so 46. A method according to any one of claims 33 to 36 and 43, wherein Remain_bit_GOP is initially calculated as the 
sum of 

a) the difference D1 between the occupancy value of stream C at the splice and the occupancy value of the ' 
nearest I or P frame of stream B following the splice; and. 

55 b) the number of bits calculated for the GOP as if the said difference a) was zero. 

47. A method according to claim 46, wherein 



40 



45 



44. A 



45. A 



13 




EP 1 045 589 A2 



JV 48* 

5 . . 



iV^v- c : 
•ii******fj^l 



Remain_bit_GOP is subsequently changed at every I and P frame. 

A method according to claim 47, wherein 

Remain_bit__GOP is subsequently changed at every I and P frame in dependence upon the difference D2 
between 



i(y 



49. 



3;;, 50. 
A 



51. 



a) the occupancy value of stream B at the current I or P frame; 

b) the occupancy value of stream B at the succeeding I or P frame. 

A method according to claim 48, wherein, if the said difference D2 is negative, Remain_bit_GOP is reduced by the 
said difference D2. 

A method according to claim 47 or 48, wherein, if the said difference D1 or D2 is positive, a value E equal to the 
difference D1 or D2 divided by the number of I and P frames remaining in the GOP is calculated and E is added to 
Remaln_bit_GOP at each subsequent I and P frame in the GOP. 

A method according to any one of claims 46 to 50, wherein the value of Remain_bit _GOP is reduced on every 
frame by the number of bits used to encode that frame. 



20 
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52. A method according to claim 42, wherein a value F is calculated as the sum of; 

a) the difference between the values of occupancy of stream C at the splice point and the prediction of the 
occupancy at the said predicted position; and 

b) the number of bits calculated for the GOP as if the said difference a) was zero, and F is apportioned between 
the said first and next GOPs. 



? 53: 
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54. 



55. 



56. 
,4, 57. 

40 

58. 



A method according to any one of claims 33 to 52, wherein the complexity values of stream B replace those of 
stream A at the splice point 

A method according to any one of claims 33 to 53, wherein the virtual buffer values of stream B replace those of 
stream A at the splice point 

A computer program product arranged to implement the method of anyone of claim 28 to 54 when run on a pro- 
grammable signal processor. 

A signal processor substantially as hereinbefore described with reference to the accompanying drawings. 

A signal processing method substantially as hereinbefore described with reference to the accompanying drawings. 
A computer program product arranged to implement the method of claim 57. 
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Display order, showing picture type decision - Long 
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Stream C! = Final Stream C (after picture type decisions) 
(1). (2), (3). (4) = Picture type decisions made at frames 

Fig. 4 
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Processing order, as shown on VBV occupancy g raphs -Uy^ 

FrameNo. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 

Stream A: I BBPBBPBBPBB I BBPBBPBBPBB 
StreamB: . . I BBPBBPBBPBB I BBPBBPBBP 

Stream C!: I BBP8BPBBPBB I BBPBBPBBPBB 



Disp C! : 201 534867 11 9 10 14 12 13 17 15 16 20 18 19 23 21 22 



Frame No. 24 25 26 27 28 

Stream A : I B B P B 
Stream B : B B I B B 
P P 

Stream C!: £ B 8 p B 



SPLICE POINT 
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VBV Lock 



Frame No. 72 73 

Stream A : . . 
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- = reuse 
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Disp C! 71 72 



FrameNo. = processing order frame no. , . 

Disp C! = display order frame no. for stream C J-Vtr p^4ure ry>e cKec\s\ons) 

Fig. 5 
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Fig. 6 
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Fig. 6 A 



VBV occupancy for short latency "long GOP - long GOP" splice 
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Fig. 6B 



VBV occupancy for short latency "long GOP - long GOP" splice 
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Fig. 6C 

VBV occupancy for short latency "long GOP - long GOP” splice 
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Fig. 6D 

VBV occupancy for short latency "long GOP - long GOP" splice 
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VBV occupancy for short latency "long GOP - long GOP" splice 




Fig. 7B 
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VBV occupancy for long latency "long GOP - long GOP” splice 



VBV occupancy Remain bit GOP 




Fig. 8A 
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VBV occupancy for long latency "long GOP - long GOP" splice 




Fig. 8B 
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